RANDOM PROJECTIONS Margin-constrained Random Projections And Very Sparse Random Projections

Abstract We1 propose methods for improving both the accuracy and efficiency of random projections, the popular dimension reduction technique in machine learning and data mining, particularly useful for estimating pairwise distances. Let A ∈ Rn×D be our n points in D dimensions. This method multiplies A by a random matrix R ∈ RD×k, reducing the D dimensions down to just k . R typically consists of i.i.d. entries in N(0, 1). The cost of the projection mapping is O(nDk). This study proposes an improved estimator of pairwise distances with provably smaller variances (errors) by taking advantage of the marginal information. We also propose very sparse random projections by replacing the N(0, 1) entries in R with entries in {−1, 0, 1} with probabilities { 1 2 √ D , 1− 1 √ D , 1 2 √ D }, for achieving a significant √ D-fold speedup, with little loss in accuracy. Previously, Achlioptas proposed sparse random projections by using entries in {−1, 0, 1} with probabilities { 1 6 , 2 3 , 1 6 }, achieving a threefold speedup.